Programación natural de un robot social mediante diálogos

La tesis tiene tres partes conceptuales diferenciadas. Una referida a los robots sociales: motivaciones para su desarrollo, aspectos de diseño y funcionamiento. Se presentan las características más importantes de Maggie, un robot social desarrollado por el RoboticsLab1, que ha servido de plataforma para el desarrollo de las otras dos partes: un sistema de gestión de diálogo y un sistema de edición verbal de secuencias. El sistema de gestión del diálogo incluye dos habilidades de voz: habilidad de reconocimiento automático del habla, y habilidad de síntesis de voz con entonación controlada. El gestor del diálogo comunica ambas habilidades entre sí, controlando el texto que se sintetiza según las entradas del reconocedor de voz. Este control se especifica en un fichero escrito en el lenguaje estándar voiceXML que se interpreta en tiempo de ejecución. Integrado con el sistema de diálogo se presenta un sistema de edición verbal de una secuencia. Una secuencia es una entidad que representa una red de movimientos del robot en interacción con su entorno, dentro del cual se ubica el usuario. El editor permite que la secuencia pueda ser editada y ejecutada sin necesidad de realizar cambios en el sistema, mediante un acceso al robot más natural y por tanto, más cercano a un usuario no experto. Las secuencias creadas cumplen con todos los requisitos especificados en el estándar de SFC's, incluyendo ejecución de acciones en serie, esquemas de seleccióne ntre varias condiciones y ramas secuenciales que se ejcutan en paralelo. También se añade la posibilidad de creación de bucles. De este modo las funcionalidades de bajo nivel del robot se hacen accesibles a alto nivel para el usuario. A modo de ejemplo, se presentan algunos resultados experimentales de edición y ejecución de secuencias mediante diálogos, tras los cuales se obtienen ciertas conclusiones. Finalmente el trabajo concluye con la propuesta de algunas líneas futuras de investigación en interacción humano robot.---------------------------------------------------------------------------------------------------------------------------------This dissertation has three different conceptual parts. One of them is related to social robotics: motivations, design aspects, and operation details. In this part, the social robot Maggie is described. This robot has been developed at the RoboticsLab and used as a platformfor the development of the two remaining .main parts of this thesis: a Dialogue Manager System and a Sequence Verbal Edition System. The Dialogue Manager System includes Borne skills such Automatic Recognition Skill and Text-to-Speech Synthesis with prosodic control. The dialogue manager communicates both skills and parses a voiceXML standard language file. A Sequence Verbal Edition System has been developed. It is integrated with the Dialogue System. A sequence can be defined as an entity that represents a net of robot movements in interaction with its environment, where the user is located. The edition system allows the edition and modification of the sequence without the necessity for making changes in the global system. Therefore, this way of operation makes the access to the robot more natural and intuitive for non-expert users. Some experimental results are presented, such Borne examples of how the sequence edition and execution processes work. Also, Borne final conclusons are extracted and drawn. The presented work gives Borne clues on how to salve Borne problems that are related to conversational interaction and natural robot programming, but this work opens more questions. Therefore, at the end of the thesis Borne human robot interaction issues are proposed as future works

End-user programming of a social robot by dialog

One of the main challenges faced by social robots is how to provide intuitive, natural and enjoyable usability for the end-user. In our ordinary environment, social robots could be important tools for education and entertainment (edutainment) in a variety of ways. This paper presents a Natural Programming System (NPS) that is geared to non-expert users. The main goal of such a system is to provide an enjoyable interactive platform for the users to build different programs within their social robot platform. The end-user can build a complex net of actions and conditions (a sequence) in a social robot via mixed-initiative dialogs and multimodal interaction. The system has been implemented and tested in Maggie, a real social robot with multiple skills, conceived as a general HRI researching platform. The robot's internal features (skills) have been implemented to be verbally accessible to the end-user, who can combine them into others that are more complex following a bottom-up model. The built sequence is internally implemented as a Sequence Function Chart (SFC), which allows parallel execution, modularity and re-use. A multimodal Dialog Manager System (DMS) takes charge of keeping the coherence of the interaction. This work is thought for bringing social robots closer to non-expert users, who can play the game of "teaching how to do things" with the robot.The research leading to these results has received funding from the RoboCity2030-II-CM project (S2009/DPI-1559), funded by Programas de Actividades I+D en la Comunidad de Madrid and cofunded by Structural Funds of the EU. The authors also gratefully acknowledge the funds provided by the Spanish Ministry of Science and Innovation (MICINN) through the project named “A New Approach to Social Robots” (AROS) DPI2008-01109

Toma de decisiones en robótica

En este artículo se presenta, en forma de tutorial, una visión de conjunto de la situación actual del problema de la toma de decisiones en robótica. El estudio se plantea de forma amplia e integradora y, por tanto, intentando no entrar en detallar soluciones concretas para problemas también concretos. El artículo está centrado sobre todo en las decisiones de alto nivel que debe tomar un robot, y no en problemas de más bajo nivel, que se solucionan empleando técnicas tradicionales de control o mediante algoritmos muy específicos. Nos referimos a "toma de decisiones" de un robot en el sentido amplio de determinar las actividades a realizar por el robot. Es decir, sin hacer ninguna exclusión a priori, basada en cuestiones tales como la estrategia de toma de decisiones empleada, el tipo de robot, las tareas que puede realizar, etc. El artículo está estructurado en una serie de secciones, en las que se tratan diversos temas de interés en robótica, desde la perspectiva de la toma de decisiones: autonomía, inteligencia, objetivos, decisiones de alto nivel, estrategias de toma de decisiones, arquitecturas de control, percepción, interacción humano-robot, aprendizaje y emociones.Este trabajo ha sido realizado parcialmente gracias al apoyo económico del Gobierno de España a través del proyecto “Interacción igual a igual humano-robot (R2H)”, del Ministerio de Educación y Ciencia, y del proyecto “Una nueva aproximación a los robots sociales (AROS)”, del Ministerio de Ciencia e Innovación. Este trabajo ha sido financiado por la Comnidad de Madrid ( S2009/DPI-1559/ROBOCITY2030 II), desarrollado por el Laboratorio de Robótica (Robotics Lab)de laUniversidad Carlos III de MadridPublicad

Augmented robotics dialog system for enhancing human-robot interaction

Augmented reality, augmented television and second screen are cutting edge technologies that provide end users extra and enhanced information related to certain events in real time. This enriched information helps users better understand such events, at the same time providing a more satisfactory experience. In the present paper, we apply this main idea to human-robot interaction (HRI), to how users and robots interchange information. The ultimate goal of this paper is to improve the quality of HRI, developing a new dialog manager system that incorporates enriched information from the semantic web. This work presents the augmented robotic dialog system (ARDS), which uses natural language understanding mechanisms to provide two features: (i) a non-grammar multimodal input (verbal and/or written) text; and (ii) a contextualization of the information conveyed in the interaction. This contextualization is achieved by information enrichment techniques that link the extracted information from the dialog with extra information about the world available in semantic knowledge bases. This enriched or contextualized information (information enrichment, semantic enhancement or contextualized information are used interchangeably in the rest of this paper) offers many possibilities in terms of HRI. For instance, it can enhance the robot's pro-activeness during a human-robot dialog (the enriched information can be used to propose new topics during the dialog, while ensuring a coherent interaction). Another possibility is to display additional multimedia content related to the enriched information on a visual device. This paper describes the ARDS and shows a proof of concept of its applications.The authors gratefully acknowledge the funds provided by the Spanish MICINN (Ministry of Science and Innovation) through the project “Aplicaciones de los robots sociales”, DPI2011-26980 from the Spanish Ministry of Economy and Competitiveness. The research leading to these results has received funding from the RoboCity2030-III-CM project (Robótica aplicada a la mejora de la calidad de vida de los ciudadanos. fase III; S2013/MIT-2748), funded by Programas de Actividades I+D en la Comunidad de Madrid and co-funded by the Structural Funds of the EU

Sound synthesis for communicating nonverbal expressive cues

Non-verbal sounds (NVS) constitute an appealing communicative channel for transmitting a message during a dialog. They provide two main benefits, such as they are not linked to any particular language, and they can express a message in a short time. NVS have been successfully used in robotics, cell phones, and science fiction films. However, there is a lack of deep studies on how to model NVS. For instance, most of the systems for NVS expression are ad hoc solutions that focus on the communication of the most prominent emotion. Only a small number of papers have proposed a more general model or dealt directly with the expression of pure communicative acts, such as affirmation, denial, or greeting. In this paper we propose a system, referred to as the sonic expression system (SES), that is able to generate NVS on the fly by adapting the sound to the context of the interaction. The system is designed to be used by social robots while conducting human robot interactions. It is based on a model that includes several acoustic features from the amplitude, frequency, and time spaces. In order to evaluate the capabilities of the system, nine categories of communicative acts were created. By means of an online questionnaire, 51 participants classified the utterances according to their meaning, such as agreement, hesitation, denial, hush, question, summon, encouragement, greetings, and laughing. The results showed how very different NVS generated by our SES can be used for communicating.Publicad

User localization during human-robot interaction

This paper presents a user localization system based on the fusion of visual information and sound source localization, implemented on a social robot called Maggie. One of the main requisites to obtain a natural interaction between human-human and human-robot is an adequate spatial situation between the interlocutors, that is, to be orientated and situated at the right distance during the conversation in order to have a satisfactory communicative process. Our social robot uses a complete multimodal dialog system which manages the user-robot interaction during the communicative process. One of its main components is the presented user localization system. To determine the most suitable allocation of the robot in relation to the user, a proxemic study of the human-robot interaction is required, which is described in this paper. The study has been made with two groups of users: children, aged between 8 and 17, and adults. Finally, at the end of the paper, experimental results with the proposed multimodal dialog system are presented.The authors gratefully acknowledge the funds provided by the Spanish Government through the project “A new approach to social robotics” (AROS), of MICINN (Ministry of Science and Innovation)

Multimodal Fusion as Communicative Acts during Human-Robot Interaction

Research on dialog systems is a very active area in social robotics. During the last two decades, these systems have evolved from those based only on speech recognition and synthesis to the current and modern systems, which include new components and multimodality. By multimodal dialogue we mean the interchange of information among several interlocutors, not just using their voice as the mean of transmission but also all the available channels such as gestures, facial expressions, touch, sounds, etc. These channels add information to the message to be transmitted in every dialogue turn. The dialogue manager (IDiM) is one of the components of the robotic dialog system (RDS) and is in charge of managing the dialogue flow during the conversational turns. In order to do that, it is necessary to coherently treat the inputs and outputs of information that flow by different communication channels: audio, vision, radio frequency, touch, etc. In our approach, this multichannel input of information is temporarily fused into communicative acts (CAs). Each CA groups the information that flows through the different input channels into the same pack, transmitting a unique message or global idea. Therefore, this temporary fusion of information allows the IDiM to abstract from the channels used during the interaction, focusing only on the message, not on the way it is transmitted. This article presents the whole RDS and the description of how the multimodal fusion of information is made as CAs. Finally, several scenarios where the multimodal dialogue is used are presented.Comunidad de Madri

Learning Behaviors by an Autonomous Social Robot with Motivations

In this study, an autonomous social robot is living in a laboratory where it can interact with several items (people included). Its goal is to learn by itself the proper behaviors in order to maintain its well-being at as high a quality as possible. Several experiments have been conducted to test the performance of the system. The Object Q-Learning algorithm has been implemented in the robot as the learning algorithm. This algorithm is a variation of the traditional Q-Learning because it considers a reduced state space and collateral effects. The comparison of the performance of both algorithms is shown in the first part of the experiments. Moreover, two mechanisms intended to reduce the learning session durations have been included: Well-Balanced Exploration and Amplified Reward. Their advantages are justified in the results obtained in the second part of the experiments. Finally, the behaviors learned by our robot are analyzed. The resulting behaviors have not been preprogrammed. In fact, they have been learned by real interaction in the real world and are related to the motivations of the robot. These are natural behaviors in the sense that they can be easily understood by humans observing the robot.The authors gratefully acknowledge the funds provided by the Spanish Government through the project call "Aplicaciones de los robots sociales", DPI2011-26980 from the Spanish Ministry of Economy and Competitiveness.Publicad

A multimodal emotion detection system during human-robot interaction

In this paper, a multimodal user-emotion detection system for social robots is presented. This system is intended to be used during human-robot interaction, and it is integrated as part of the overall interaction system of the robot: the Robotics Dialog System (RDS). Two modes are used to detect emotions: the voice and face expression analysis. In order to analyze the voice of the user, a new component has been developed: Gender and Emotion Voice Analysis (GEVA), which is written using the Chuck language. For emotion detection in facial expressions, the system, Gender and Emotion Facial Analysis (GEFA), has been also developed. This last system integrates two third-party solutions: Sophisticated High-speed Object Recognition Engine (SHORE) and Computer Expression Recognition Toolbox (CERT). Once these new components (GEVA and GEFA) give their results, a decision rule is applied in order to combine the information given by both of them. The result of this rule, the detected emotion, is integrated into the dialog system through communicative acts. Hence, each communicative act gives, among other things, the detected emotion of the user to the RDS so it can adapt its strategy in order to get a greater satisfaction degree during the human-robot dialog. Each of the new components, GEVA and GEFA, can also be used individually. Moreover, they are integrated with the robotic control platform ROS (Robot Operating System). Several experiments with real users were performed to determine the accuracy of each component and to set the final decision rule. The results obtained from applying this decision rule in these experiments show a high success rate in automatic user emotion recognition, improving the results given by the two information channels (audio and visual) separately.The authors gratefully acknowledge the funds provided by the Spanish MICINN (Ministry of Science and Innovation) through the project “Aplicaciones de los robots sociales”, DPI2011-26980 from the Spanish Ministry of Economy and Competitiveness. Moreover, the research leading to these results has received funding from the RoboCity2030-II-CM project (S2009/DPI-1559), funded by Programas de Actividades I+D en la Comunidad de Madrid and cofunded by Structural Funds of the EU